publication . Preprint . Article . 2017

Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models

Carlos Outeiral Rubiera;
Open Access English
  • Published: 30 May 2017
Abstract
In unsupervised data generation tasks, besides the generation of a sample based on previous observations, one would often like to give hints to the model in order to bias the generation towards desirable metrics. We propose a method that combines Generative Adversarial Networks (GANs) and reinforcement learning (RL) in order to accomplish exactly that. While RL biases the data generation process towards arbitrary metrics, the GAN component of the reward function ensures that the model still remembers information learned from data. We build upon previous results that incorporated GANs and RL in order to generate sequence data and test this model in several settin...
Subjects
free text keywords: Statistics - Machine Learning, Computer Science - Learning
19 references, page 1 of 2

[2] J.-M. I. Amaury Habrard and M. S. David Rizo. Melody recognition with learned edit distances. Structural, Syntactic, and Statistical Pattern Recognition, 2008. doi: 10.1007/ 978-3-540-89689-0_13. [OpenAIRE]

[3] M. E. Bonds. A History of Music in Western Culture. page 123, 2006.

[4] P. Ertl and A. Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of Cheminformatics, 1, 2009. doi: 10.1186/1758-2946-1-8.

[5] G. Ewer. The Power of the Melodic Leap. 2013. URL http://www.secretsofsongwriting. com/2013/01/16/the-power-of-the-melodic-leap/.

[6] R. Gómez-Bombarelli, J. Aguilera-Iparraguirre, T. D. Hirzel, D. Duvenaud, D. Maclaurin, M. A. Blood-Forsythe, H. S. Chae, M. Einzinger, D.-G. Ha, G. M. Tony Wu, S. Jeon, H. Kang, H. Miyazaki, M. Numata, S. Kim, W. Huang, S. I. Hong, M. Baldo, R. P. Adams, and A. AspuruGuzik. Design of efficient molecular organic light-emitting diodes by a high-throughput virtual screening and experimental approach. Nature Materials, 15(10):1120-1127, aug 2016. doi: 10.1038/nmat4717. [OpenAIRE]

[7] R. Gómez-Bombarelli, D. Duvenaud, J. M. Hernández-Lobato, J. Aguilera-Iparraguirre, T. D. Hirzel, R. P. Adams, and A. Aspuru-Guzik. Automatic chemical design using a data-driven continuous representation of molecules. 2016. arXiv:1610.02415 [cs.LG]. [OpenAIRE]

[8] I. Goodfellow, J. Pouget-Abadie, and M. Mirza. Generative Adversarial Networks. 2014. arXiv:1406.2661 [stat.ML].

[9] J. Hachmann, R. Olivares-Amaya, A. Jinich, A. L. Appleton, M. A. Blood-Forsythe, L. R. Seress, C. Román-Salgado, K. Trepte, S. Atahan-Evrenk, S. Er, S. Shrestha, R. Mondal, A. Sokolov, Z. Bao, and A. Aspuru-Guzik. Lead candidates for high-performance organic photovoltaics from high-throughput quantum chemistry - the harvard clean energy project. Energy Environ. Sci., 7(2):698-704, 2014. doi: 10.1039/c3ee42756k. [OpenAIRE]

[10] J. Hochreiter, Sepp and Schmidhuber and Urgen. Long Short Term Memory. Memory, 9(1993): 1-28, 1996. arxiv:1206.2944.

[11] J. J. Irwin and B. K. Shoichet. ZINC - A Free Database of Commercially Available Compounds for Virtual Screening. J Chem Inf Model, 2006.

[12] C. A. James, R. Apodaca, N. O'Boyle, A. Dalke, J. van Drie, P. Ertl, G. Hutchison, G. Landrum, C. Morley, E. Willighagen, H. D. Winter, T. Vandermeersch, and J. May. OpenSMILES specification. 2016. URL http://opensmiles.org/.

[13] N. Jaques, S. Gu, D. Bahdanau, J. Miguel, H. Lobato, R. E. Turner, and D. Eck. Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control. 2017. arXiv:1611.02796 [cs.LG].

[14] D. P. Kingma and J. Ba. Adam: A Method for Stochastic Optimization. ICLR, 2014. arXiv:1412.6980 [cs.LG].

[15] G. Landrum. Rdkit: Open-source cheminformatics. URL http://www.rdkit.org.

[16] O. Legname. Density Degree of Intervals and Chords. 1998. URL http://www.oneonta. edu/faculty/legnamo/theorist/density/density.html.

19 references, page 1 of 2
Abstract
In unsupervised data generation tasks, besides the generation of a sample based on previous observations, one would often like to give hints to the model in order to bias the generation towards desirable metrics. We propose a method that combines Generative Adversarial Networks (GANs) and reinforcement learning (RL) in order to accomplish exactly that. While RL biases the data generation process towards arbitrary metrics, the GAN component of the reward function ensures that the model still remembers information learned from data. We build upon previous results that incorporated GANs and RL in order to generate sequence data and test this model in several settin...
Subjects
free text keywords: Statistics - Machine Learning, Computer Science - Learning
19 references, page 1 of 2

[2] J.-M. I. Amaury Habrard and M. S. David Rizo. Melody recognition with learned edit distances. Structural, Syntactic, and Statistical Pattern Recognition, 2008. doi: 10.1007/ 978-3-540-89689-0_13. [OpenAIRE]

[3] M. E. Bonds. A History of Music in Western Culture. page 123, 2006.

[4] P. Ertl and A. Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of Cheminformatics, 1, 2009. doi: 10.1186/1758-2946-1-8.

[5] G. Ewer. The Power of the Melodic Leap. 2013. URL http://www.secretsofsongwriting. com/2013/01/16/the-power-of-the-melodic-leap/.

[6] R. Gómez-Bombarelli, J. Aguilera-Iparraguirre, T. D. Hirzel, D. Duvenaud, D. Maclaurin, M. A. Blood-Forsythe, H. S. Chae, M. Einzinger, D.-G. Ha, G. M. Tony Wu, S. Jeon, H. Kang, H. Miyazaki, M. Numata, S. Kim, W. Huang, S. I. Hong, M. Baldo, R. P. Adams, and A. AspuruGuzik. Design of efficient molecular organic light-emitting diodes by a high-throughput virtual screening and experimental approach. Nature Materials, 15(10):1120-1127, aug 2016. doi: 10.1038/nmat4717. [OpenAIRE]

[7] R. Gómez-Bombarelli, D. Duvenaud, J. M. Hernández-Lobato, J. Aguilera-Iparraguirre, T. D. Hirzel, R. P. Adams, and A. Aspuru-Guzik. Automatic chemical design using a data-driven continuous representation of molecules. 2016. arXiv:1610.02415 [cs.LG]. [OpenAIRE]

[8] I. Goodfellow, J. Pouget-Abadie, and M. Mirza. Generative Adversarial Networks. 2014. arXiv:1406.2661 [stat.ML].

[9] J. Hachmann, R. Olivares-Amaya, A. Jinich, A. L. Appleton, M. A. Blood-Forsythe, L. R. Seress, C. Román-Salgado, K. Trepte, S. Atahan-Evrenk, S. Er, S. Shrestha, R. Mondal, A. Sokolov, Z. Bao, and A. Aspuru-Guzik. Lead candidates for high-performance organic photovoltaics from high-throughput quantum chemistry - the harvard clean energy project. Energy Environ. Sci., 7(2):698-704, 2014. doi: 10.1039/c3ee42756k. [OpenAIRE]

[10] J. Hochreiter, Sepp and Schmidhuber and Urgen. Long Short Term Memory. Memory, 9(1993): 1-28, 1996. arxiv:1206.2944.

[11] J. J. Irwin and B. K. Shoichet. ZINC - A Free Database of Commercially Available Compounds for Virtual Screening. J Chem Inf Model, 2006.

[12] C. A. James, R. Apodaca, N. O'Boyle, A. Dalke, J. van Drie, P. Ertl, G. Hutchison, G. Landrum, C. Morley, E. Willighagen, H. D. Winter, T. Vandermeersch, and J. May. OpenSMILES specification. 2016. URL http://opensmiles.org/.

[13] N. Jaques, S. Gu, D. Bahdanau, J. Miguel, H. Lobato, R. E. Turner, and D. Eck. Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control. 2017. arXiv:1611.02796 [cs.LG].

[14] D. P. Kingma and J. Ba. Adam: A Method for Stochastic Optimization. ICLR, 2014. arXiv:1412.6980 [cs.LG].

[15] G. Landrum. Rdkit: Open-source cheminformatics. URL http://www.rdkit.org.

[16] O. Legname. Density Degree of Intervals and Chords. 1998. URL http://www.oneonta. edu/faculty/legnamo/theorist/density/density.html.

19 references, page 1 of 2
Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue
publication . Preprint . Article . 2017

Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models

Carlos Outeiral Rubiera;