publication . Preprint . Conference object . Part of book or chapter of book . 2018

Real-Time Marker-Less Multi-person 3D Pose Estimation in RGB-Depth Camera Networks

Marco Carraro; Matteo Munaro; Jeff Burke; Emanuele Menegatti;
Open Access English
  • Published: 31 Dec 2018
  • Country: Italy
Abstract
This paper proposes a novel system to estimate and track the 3D poses of multiple persons in calibrated RGB-Depth camera networks. The multi-view 3D pose of each person is computed by a central node which receives the single-view outcomes from each camera of the network. Each single-view outcome is computed by using a CNN for 2D pose estimation and extending the resulting skeletons to 3D by means of the sensor depth. The proposed system is marker-less, multi-person, independent of background and does not make any assumption on people appearance and initial pose. The system provides real-time outcomes, thus being perfectly suited for applications requiring user i...
Subjects
free text keywords: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Robotics, Control and Systems Engineering; Computer Science (all), Computer vision, Pose, Source code, media_common.quotation_subject, media_common, 3D pose estimation, Central node, Computer science, RGB color model, Artificial intelligence, business.industry, business, Camera network
30 references, page 1 of 2

[1] F. Han, X. Yang, C. Reardon, Y. Zhang, and H. Zhang, “Simultaneous feature and body-part learning for real-time robot awareness of human behaviors,” 2017.

[2] M. Zanfir, M. Leordeanu, and C. Sminchisescu, “The moving pose: An efficient 3d kinematics descriptor for low-latency action recognition and detection,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 2752-2759, 2013.

[3] C. Wang, Y. Wang, and A. L. Yuille, “An approach to pose-based action recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 915-922, 2013.

[4] S. Ghidoni and M. Munaro, “A multi-viewpoint feature-based reidentification system driven by skeleton keypoints,” Robotics and Autonomous Systems, vol. 90, pp. 45-54, 2017. [OpenAIRE]

[5] A. Jaimes and N. Sebe, “Multimodal human-computer interaction: A survey,” Computer vision and image understanding, vol. 108, no. 1, pp. 116-134, 2007. [OpenAIRE]

[6] C. Morato, K. N. Kaipa, B. Zhao, and S. K. Gupta, “Toward safe human robot collaboration by using multiple kinects based real-time human tracking,” Journal of Computing and Information Science in Engineering, vol. 14, no. 1, p. 011006, 2014.

[7] S. Michieletto, F. Stival, F. Castelli, M. Khosravi, A. Landini, S. Ellero, R. LandÚ, N. Boscolo, S. Tonello, B. Varaticeanu, C. Nicolescu, and E. Pagello, “Flexicoil: Flexible robotized coils winding for electric machines manufacturing industry,” in ICRA workshop on Industry of the future: Collaborative, Connected, Cognitive, 2017.

[8] F. Stival, S. Michieletto, and E. Pagello, “How to deploy a wire with a robotic platform: Learning from human visual demonstrations,” in FAIM 2017, 2017. [OpenAIRE]

[9] Z. Zivkovic, “Wireless smart camera network for real-time human 3d pose reconstruction,” Computer Vision and Image Understanding, vol. 114, no. 11, pp. 1215-1222, 2010.

[10] M. Carraro, M. Munaro, and E. Menegatti, “A powerful and costefficient human perception system for camera networks and mobile robotics,” in International Conference on Intelligent Autonomous Systems, pp. 485-497, Springer, Cham, 2016.

[11] M. Carraro, M. Munaro, and E. Menegatti, “Cost-efficient rgb-d smart camera for people detection and tracking,” Journal of Electronic Imaging, vol. 25, no. 4, pp. 041007-041007, 2016. [OpenAIRE]

[12] F. Basso, R. Levorato, and E. Menegatti, “Online calibration for networks of cameras and depth sensors,” in OMNIVIS: The 12th Workshop on Non-classical Cameras, Camera Networks and Omnidirectional Vision-2014 IEEE International Conference on Robotics and Automation (ICRA 2014), 2014.

[13] S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh, “Convolutional pose machines,” in CVPR, 2016.

[14] Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields,” in CVPR, 2017.

[15] M. Munaro, A. Horn, R. Illum, J. Burke, and R. B. Rusu, “Openptrack: People tracking for heterogeneous networks of color-depth cameras,” in IAS-13 Workshop Proceedings: 1st Intl. Workshop on 3D Robot Perception with Point Cloud Library, pp. 235-247, 2014.

30 references, page 1 of 2
Abstract
This paper proposes a novel system to estimate and track the 3D poses of multiple persons in calibrated RGB-Depth camera networks. The multi-view 3D pose of each person is computed by a central node which receives the single-view outcomes from each camera of the network. Each single-view outcome is computed by using a CNN for 2D pose estimation and extending the resulting skeletons to 3D by means of the sensor depth. The proposed system is marker-less, multi-person, independent of background and does not make any assumption on people appearance and initial pose. The system provides real-time outcomes, thus being perfectly suited for applications requiring user i...
Subjects
free text keywords: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Robotics, Control and Systems Engineering; Computer Science (all), Computer vision, Pose, Source code, media_common.quotation_subject, media_common, 3D pose estimation, Central node, Computer science, RGB color model, Artificial intelligence, business.industry, business, Camera network
30 references, page 1 of 2

[1] F. Han, X. Yang, C. Reardon, Y. Zhang, and H. Zhang, “Simultaneous feature and body-part learning for real-time robot awareness of human behaviors,” 2017.

[2] M. Zanfir, M. Leordeanu, and C. Sminchisescu, “The moving pose: An efficient 3d kinematics descriptor for low-latency action recognition and detection,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 2752-2759, 2013.

[3] C. Wang, Y. Wang, and A. L. Yuille, “An approach to pose-based action recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 915-922, 2013.

[4] S. Ghidoni and M. Munaro, “A multi-viewpoint feature-based reidentification system driven by skeleton keypoints,” Robotics and Autonomous Systems, vol. 90, pp. 45-54, 2017. [OpenAIRE]

[5] A. Jaimes and N. Sebe, “Multimodal human-computer interaction: A survey,” Computer vision and image understanding, vol. 108, no. 1, pp. 116-134, 2007. [OpenAIRE]

[6] C. Morato, K. N. Kaipa, B. Zhao, and S. K. Gupta, “Toward safe human robot collaboration by using multiple kinects based real-time human tracking,” Journal of Computing and Information Science in Engineering, vol. 14, no. 1, p. 011006, 2014.

[7] S. Michieletto, F. Stival, F. Castelli, M. Khosravi, A. Landini, S. Ellero, R. LandÚ, N. Boscolo, S. Tonello, B. Varaticeanu, C. Nicolescu, and E. Pagello, “Flexicoil: Flexible robotized coils winding for electric machines manufacturing industry,” in ICRA workshop on Industry of the future: Collaborative, Connected, Cognitive, 2017.

[8] F. Stival, S. Michieletto, and E. Pagello, “How to deploy a wire with a robotic platform: Learning from human visual demonstrations,” in FAIM 2017, 2017. [OpenAIRE]

[9] Z. Zivkovic, “Wireless smart camera network for real-time human 3d pose reconstruction,” Computer Vision and Image Understanding, vol. 114, no. 11, pp. 1215-1222, 2010.

[10] M. Carraro, M. Munaro, and E. Menegatti, “A powerful and costefficient human perception system for camera networks and mobile robotics,” in International Conference on Intelligent Autonomous Systems, pp. 485-497, Springer, Cham, 2016.

[11] M. Carraro, M. Munaro, and E. Menegatti, “Cost-efficient rgb-d smart camera for people detection and tracking,” Journal of Electronic Imaging, vol. 25, no. 4, pp. 041007-041007, 2016. [OpenAIRE]

[12] F. Basso, R. Levorato, and E. Menegatti, “Online calibration for networks of cameras and depth sensors,” in OMNIVIS: The 12th Workshop on Non-classical Cameras, Camera Networks and Omnidirectional Vision-2014 IEEE International Conference on Robotics and Automation (ICRA 2014), 2014.

[13] S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh, “Convolutional pose machines,” in CVPR, 2016.

[14] Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields,” in CVPR, 2017.

[15] M. Munaro, A. Horn, R. Illum, J. Burke, and R. B. Rusu, “Openptrack: People tracking for heterogeneous networks of color-depth cameras,” in IAS-13 Workshop Proceedings: 1st Intl. Workshop on 3D Robot Perception with Point Cloud Library, pp. 235-247, 2014.

30 references, page 1 of 2
Powered by OpenAIRE Research Graph
Any information missing or wrong?Report an Issue