publication . Preprint . 2019

Open-Sourced Reinforcement Learning Environments for Surgical Robotics

Richter, Florian; Orosco, Ryan K.; Yip, Michael C.;
Open Access English
  • Published: 05 Mar 2019
Abstract
Reinforcement Learning (RL) is a machine learning framework for artificially intelligent systems to solve a variety of complex problems. Recent years has seen a surge of successes solving challenging games and smaller domain problems, including simple though non-specific robotic manipulation and grasping tasks. Rapid successes in RL have come in part due to the strong collaborative effort by the RL community to work on common, open-sourced environment simulators such as OpenAI's Gym that allow for expedited development and valid comparisons between different, state-of-art strategies. In this paper, we aim to start the bridge between the RL and the surgical robot...
Subjects
free text keywords: Computer Science - Robotics
Download from
35 references, page 1 of 3

[1] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press, 2018.

[2] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013. [OpenAIRE]

[3] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al., “Mastering the game of go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, p. 484, 2016.

[4] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 23-30, IEEE, 2017.

[5] J. Tobin, L. Biewald, R. Duan, M. Andrychowicz, A. Handa, V. Kumar, B. McGrew, A. Ray, J. Schneider, P. Welinder, et al., “Domain randomization and generative models for robotic grasping,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3482-3489, IEEE, 2018. [OpenAIRE]

[6] A. J. Hung, J. Chen, D. H. Anthony Jarc, H. Djaladat, and I. S. Gilla, “Development and validation of objective performance metrics for robot-assisted radical prostatectomy: A pilot study,” The Journal of Urology, vol. 199, pp. 296-304, Jan 2018.

[7] F. Richter, R. K. Orosco, and M. C. Yip, “Motion scaling solutions for improved performance in high delay surgical teleoperation,” arXiv preprint arXiv:1902.03290, 2019. [OpenAIRE]

[8] F. Richter, Y. Zhang, Y. Zhi, R. K. Orosco, and M. C. Yip, “Augmented reality predictive displays to help mitigate the effects of delayed telesurgery,” arXiv preprint arXiv:1809.08627, 2018.

[9] M. Yip and N. Das, ROBOT AUTONOMY FOR SURGERY, ch. Chapter 10, pp. 281-313.

[10] T. Osa, N. Sugita, and M. Mitsuishi, “Online trajectory planning in dynamic environments for surgical task automation.,” in Robotics: Science and Systems, pp. 1-9, 2014.

[11] J. Van Den Berg, S. Miller, D. Duckworth, H. Hu, A. Wan, X.- Y. Fu, K. Goldberg, and P. Abbeel, “Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations,” in 2010 IEEE International Conference on Robotics and Automation (ICRA), pp. 2074-2081, IEEE, 2010.

[12] J. J. Ji, S. Krishnan, V. Patel, D. Fer, and K. Goldberg, “Learning 2d surgical camera motion from demonstrations,” in 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE), pp. 35-42, IEEE, 2018.

[13] O. Weede, H. Mo¨nnich, B. Mu¨ller, and H. Wo¨rn, “An intelligent and autonomous endoscopic guidance system for minimally invasive surgery,” in 2011 IEEE International Conference on Robotics and Automation, pp. 5762-5768, IEEE, 2011. [OpenAIRE]

[14] B. Thananjeyan, A. Garg, S. Krishnan, C. Chen, L. Miller, and K. Goldberg, “Multilateral surgical pattern cutting in 2d orthotropic gauze with deep reinforcement learning policies for tensioning,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 2371-2378, IEEE, 2017.

[15] A. Murali, S. Sen, B. Kehoe, A. Garg, S. McFarland, S. Patil, W. D. Boyd, S. Lim, P. Abbeel, and K. Goldberg, “Learning by observation for surgical subtasks: Multilateral cutting of 3d viscoelastic and 2d orthotropic tissue phantoms,” in 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 1202-1209, IEEE, 2015.

35 references, page 1 of 3
Abstract
Reinforcement Learning (RL) is a machine learning framework for artificially intelligent systems to solve a variety of complex problems. Recent years has seen a surge of successes solving challenging games and smaller domain problems, including simple though non-specific robotic manipulation and grasping tasks. Rapid successes in RL have come in part due to the strong collaborative effort by the RL community to work on common, open-sourced environment simulators such as OpenAI's Gym that allow for expedited development and valid comparisons between different, state-of-art strategies. In this paper, we aim to start the bridge between the RL and the surgical robot...
Subjects
free text keywords: Computer Science - Robotics
Download from
35 references, page 1 of 3

[1] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press, 2018.

[2] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013. [OpenAIRE]

[3] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al., “Mastering the game of go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, p. 484, 2016.

[4] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 23-30, IEEE, 2017.

[5] J. Tobin, L. Biewald, R. Duan, M. Andrychowicz, A. Handa, V. Kumar, B. McGrew, A. Ray, J. Schneider, P. Welinder, et al., “Domain randomization and generative models for robotic grasping,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3482-3489, IEEE, 2018. [OpenAIRE]

[6] A. J. Hung, J. Chen, D. H. Anthony Jarc, H. Djaladat, and I. S. Gilla, “Development and validation of objective performance metrics for robot-assisted radical prostatectomy: A pilot study,” The Journal of Urology, vol. 199, pp. 296-304, Jan 2018.

[7] F. Richter, R. K. Orosco, and M. C. Yip, “Motion scaling solutions for improved performance in high delay surgical teleoperation,” arXiv preprint arXiv:1902.03290, 2019. [OpenAIRE]

[8] F. Richter, Y. Zhang, Y. Zhi, R. K. Orosco, and M. C. Yip, “Augmented reality predictive displays to help mitigate the effects of delayed telesurgery,” arXiv preprint arXiv:1809.08627, 2018.

[9] M. Yip and N. Das, ROBOT AUTONOMY FOR SURGERY, ch. Chapter 10, pp. 281-313.

[10] T. Osa, N. Sugita, and M. Mitsuishi, “Online trajectory planning in dynamic environments for surgical task automation.,” in Robotics: Science and Systems, pp. 1-9, 2014.

[11] J. Van Den Berg, S. Miller, D. Duckworth, H. Hu, A. Wan, X.- Y. Fu, K. Goldberg, and P. Abbeel, “Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations,” in 2010 IEEE International Conference on Robotics and Automation (ICRA), pp. 2074-2081, IEEE, 2010.

[12] J. J. Ji, S. Krishnan, V. Patel, D. Fer, and K. Goldberg, “Learning 2d surgical camera motion from demonstrations,” in 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE), pp. 35-42, IEEE, 2018.

[13] O. Weede, H. Mo¨nnich, B. Mu¨ller, and H. Wo¨rn, “An intelligent and autonomous endoscopic guidance system for minimally invasive surgery,” in 2011 IEEE International Conference on Robotics and Automation, pp. 5762-5768, IEEE, 2011. [OpenAIRE]

[14] B. Thananjeyan, A. Garg, S. Krishnan, C. Chen, L. Miller, and K. Goldberg, “Multilateral surgical pattern cutting in 2d orthotropic gauze with deep reinforcement learning policies for tensioning,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 2371-2378, IEEE, 2017.

[15] A. Murali, S. Sen, B. Kehoe, A. Garg, S. McFarland, S. Patil, W. D. Boyd, S. Lim, P. Abbeel, and K. Goldberg, “Learning by observation for surgical subtasks: Multilateral cutting of 3d viscoelastic and 2d orthotropic tissue phantoms,” in 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 1202-1209, IEEE, 2015.

35 references, page 1 of 3
Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue