publication . Conference object . Preprint . 2017

SceneCut: Joint Geometric and Object Segmentation for Indoor Scenes

Pham, Trung; Do, Thanh-Toan; Sünderhauf, Niko; Reid, Ian;
Open Access
  • Published: 21 Sep 2017
  • Publisher: IEEE
Abstract
This paper presents SceneCut, a novel approach to jointly discover previously unseen objects and non-object surfaces using a single RGB-D image. SceneCut's joint reasoning over scene semantics and geometry allows a robot to detect and segment object instances in complex scenes where modern deep learning-based methods either fail to separate object instances, or fail to detect objects that were not seen during training. SceneCut automatically decomposes a scene into meaningful regions which either represent objects or scene surfaces. The decomposition is qualified by an unified energy function over objectness and geometric fitting. We show how this energy functio...
Subjects
ACM Computing Classification System: ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION
free text keywords: Segmentation, Geometric fitting, Engineering, business.industry, business, Computer vision, Hierarchy, Semantics, Control engineering, RGB color model, Artificial intelligence, Image segmentation, Deep learning, Robot, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Robotics
Funded by
ARC| ARC Centres of Excellences - Grant ID: CE140100016
Project
  • Funder: Australian Research Council (ARC) (ARC)
  • Project Code: CE140100016
  • Funding stream: ARC Centres of Excellences
,
ARC| Australian Laureate Fellowships - Grant ID: FL130100102
Project
  • Funder: Australian Research Council (ARC) (ARC)
  • Project Code: FL130100102
  • Funding stream: Australian Laureate Fellowships
26 references, page 1 of 2

[1] G. Lin, A. Milan, C. Shen, and I. Reid, “RefineNet: Multi-path refinement networks for high-resolution semantic segmentation,” in CVPR, Jul. 2017.

[2] K. He, G. Gkioxari, P. Dolla´r, and R. Girshick, “Mask R-CNN,” arXiv preprint arXiv:1703.06870, 2017.

[3] J. Redmon and A. Farhadi, “YOLO9000: better, faster, stronger,” CoRR, vol. abs/1612.08242, 2016.

[4] W. J. Scheirer, A. de Rezende Rocha, A. Sapkota, and T. E. Boult, “Toward open set recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 7, pp. 1757-1772, 2013.

[5] C. Cadena, L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza, J. Neira, I. Reid, and J. J. Leonard, “Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age,” IEEE Transactions on Robotics, vol. 32, no. 6, pp. 1309-1332, 2016.

[6] R. F. Salas-Moreno, R. A. Newcombe, H. Strasdat, P. H. J. Kelly, and A. J. Davison, “Slam++: Simultaneous localisation and mapping at the level of objects,” in 2013 IEEE Conference on Computer Vision and Pattern Recognition, June 2013, pp. 1352-1359. [OpenAIRE]

[7] N. S u¨nderhauf, T.-T. Pham, Y. Latif, M. Milford, and I. D. Reid, “Meaningful maps with object-oriented semantic mapping,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017.

[8] A. J. B. Trevor, S. Gedikli, R. B. Rusu, and H. I. Christensen, “Efficient organized point cloud segmentation with connected components,” 2013.

[9] T. T. Pham, M. Eich, I. D. Reid, and G. Wyeth, “Geometrically consistent plane extraction for dense indoor 3d maps segmentation,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016, pp. 4199-4204.

[10] N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, “Indoor segmentation and support inference from rgbd images,” in Proceedings of the 12th European Conference on Computer Vision - Volume Part V, ser. ECCV'12. Berlin, Heidelberg: Springer-Verlag, 2012, pp. 746-760. [OpenAIRE]

[11] P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 5, pp. 898-916, May 2011.

[12] K. Maninis, J. Pont-Tuset, P. Arbela´ez, and L. V. Gool, “Convolutional oriented boundaries: From image segmentation to high-level tasks,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2017.

[13] S. Gupta, P. Arbelez, and J. Malik, “Perceptual organization and recognition of indoor scenes from rgb-d images,” in 2013 IEEE Conference on Computer Vision and Pattern Recognition, June 2013, pp. 564-571.

[14] J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A benchmark for the evaluation of rgb-d slam systems,” in Proc. of the International Conference on Intelligent Robot Systems (IROS), Oct. 2012.

[15] J. McCormac, A. Handa, A. Davison, and S. Leutenegger, “Semanticfusion: Dense 3d semantic mapping with convolutional neural networks,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), May 2017, pp. 4628-4635. [OpenAIRE]

26 references, page 1 of 2
Abstract
This paper presents SceneCut, a novel approach to jointly discover previously unseen objects and non-object surfaces using a single RGB-D image. SceneCut's joint reasoning over scene semantics and geometry allows a robot to detect and segment object instances in complex scenes where modern deep learning-based methods either fail to separate object instances, or fail to detect objects that were not seen during training. SceneCut automatically decomposes a scene into meaningful regions which either represent objects or scene surfaces. The decomposition is qualified by an unified energy function over objectness and geometric fitting. We show how this energy functio...
Subjects
ACM Computing Classification System: ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION
free text keywords: Segmentation, Geometric fitting, Engineering, business.industry, business, Computer vision, Hierarchy, Semantics, Control engineering, RGB color model, Artificial intelligence, Image segmentation, Deep learning, Robot, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Robotics
Funded by
ARC| ARC Centres of Excellences - Grant ID: CE140100016
Project
  • Funder: Australian Research Council (ARC) (ARC)
  • Project Code: CE140100016
  • Funding stream: ARC Centres of Excellences
,
ARC| Australian Laureate Fellowships - Grant ID: FL130100102
Project
  • Funder: Australian Research Council (ARC) (ARC)
  • Project Code: FL130100102
  • Funding stream: Australian Laureate Fellowships
26 references, page 1 of 2

[1] G. Lin, A. Milan, C. Shen, and I. Reid, “RefineNet: Multi-path refinement networks for high-resolution semantic segmentation,” in CVPR, Jul. 2017.

[2] K. He, G. Gkioxari, P. Dolla´r, and R. Girshick, “Mask R-CNN,” arXiv preprint arXiv:1703.06870, 2017.

[3] J. Redmon and A. Farhadi, “YOLO9000: better, faster, stronger,” CoRR, vol. abs/1612.08242, 2016.

[4] W. J. Scheirer, A. de Rezende Rocha, A. Sapkota, and T. E. Boult, “Toward open set recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 7, pp. 1757-1772, 2013.

[5] C. Cadena, L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza, J. Neira, I. Reid, and J. J. Leonard, “Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age,” IEEE Transactions on Robotics, vol. 32, no. 6, pp. 1309-1332, 2016.

[6] R. F. Salas-Moreno, R. A. Newcombe, H. Strasdat, P. H. J. Kelly, and A. J. Davison, “Slam++: Simultaneous localisation and mapping at the level of objects,” in 2013 IEEE Conference on Computer Vision and Pattern Recognition, June 2013, pp. 1352-1359. [OpenAIRE]

[7] N. S u¨nderhauf, T.-T. Pham, Y. Latif, M. Milford, and I. D. Reid, “Meaningful maps with object-oriented semantic mapping,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017.

[8] A. J. B. Trevor, S. Gedikli, R. B. Rusu, and H. I. Christensen, “Efficient organized point cloud segmentation with connected components,” 2013.

[9] T. T. Pham, M. Eich, I. D. Reid, and G. Wyeth, “Geometrically consistent plane extraction for dense indoor 3d maps segmentation,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016, pp. 4199-4204.

[10] N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, “Indoor segmentation and support inference from rgbd images,” in Proceedings of the 12th European Conference on Computer Vision - Volume Part V, ser. ECCV'12. Berlin, Heidelberg: Springer-Verlag, 2012, pp. 746-760. [OpenAIRE]

[11] P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 5, pp. 898-916, May 2011.

[12] K. Maninis, J. Pont-Tuset, P. Arbela´ez, and L. V. Gool, “Convolutional oriented boundaries: From image segmentation to high-level tasks,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2017.

[13] S. Gupta, P. Arbelez, and J. Malik, “Perceptual organization and recognition of indoor scenes from rgb-d images,” in 2013 IEEE Conference on Computer Vision and Pattern Recognition, June 2013, pp. 564-571.

[14] J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A benchmark for the evaluation of rgb-d slam systems,” in Proc. of the International Conference on Intelligent Robot Systems (IROS), Oct. 2012.

[15] J. McCormac, A. Handa, A. Davison, and S. Leutenegger, “Semanticfusion: Dense 3d semantic mapping with convolutional neural networks,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), May 2017, pp. 4628-4635. [OpenAIRE]

26 references, page 1 of 2
Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue
publication . Conference object . Preprint . 2017

SceneCut: Joint Geometric and Object Segmentation for Indoor Scenes

Pham, Trung; Do, Thanh-Toan; Sünderhauf, Niko; Reid, Ian;