publication . Other literature type . Part of book or chapter of book . Preprint . 2018

Learning Energy Based Inpainting for Optical Flow

Christoph Vogel; Patrick Knöbelreiter; Thomas Pock;
  • Published: 08 Nov 2018
  • Publisher: Springer International Publishing
Abstract
Modern optical flow methods are often composed of a cascade of many independent steps or formulated as a black box neural network that is hard to interpret and analyze. In this work we seek for a plain, interpretable, but learnable solution. We propose a novel inpainting based algorithm that approaches the problem in three steps: feature selection and matching, selection of supporting points and energy based inpainting. To facilitate the inference we propose an optimization layer that allows to backpropagate through 10K iterations of a first-order method without any numerical or memory problems. Compared to recent state-of-the-art networks, our modular CNN is ve...
Subjects
free text keywords: Optical flow, Feature selection, Black box (phreaking), Modular design, business.industry, business, Inpainting, Pattern recognition, Inference, Deep learning, Artificial neural network, Artificial intelligence, Computer science, Computer Science - Computer Vision and Pattern Recognition
Funded by
EC| HOMOVIS
Project
HOMOVIS
High-level Prior Models for Computer Vision
  • Funder: European Commission (EC)
  • Project Code: 640156
  • Funding stream: H2020 | ERC | ERC-STG
41 references, page 1 of 3

1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. PAMI (2012) [OpenAIRE]

2. Agresti, G., Minto, L., Marin, G., Zanuttigh, P.: Deep learning for con dence information in stereo and tof data fusion. In: The IEEE International Conference on Computer Vision (ICCV) Workshops (Oct 2017)

3. Barron, J.T., Poole, B.: The fast bilateral solver. ECCV (2016)

4. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Img. Sci. (2009) [OpenAIRE]

5. Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sciences (2010) [OpenAIRE]

6. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical ow estimation based on a theory for warping. In: ECCV (2004)

7. Butler, D.J., Wul , J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical ow evaluation. In: A. Fitzgibbon et al. (Eds.) (ed.) ECCV. pp. 611{625. Part IV, LNCS 7577, Springer-Verlag (Oct 2012)

8. Chen, T., Xu, B., Zhang, C., Guestrin, C.: Training deep nets with sublinear memory cost. CoRR abs/1604.06174 (2016)

9. Chen, Y., Ranftl, R., Pock, T.: A bi-level view of inpainting - based image compression. CoRR abs/1401.4112 (2014) [OpenAIRE]

10. Dollar, P., Zitnick, C.L.: Structured forests for fast edge detection. In: CVPR. pp. 1841{1848. ICCV '13, IEEE (2013) [OpenAIRE]

11. Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Haz rbas, C., Golkov, V., v.d. Smagt, P., Cremers, D., Brox, T.: Flownet: Learning optical ow with convolutional networks. In: ICCV (2015)

12. Galic, I., Weickert, J., Welk, M., Bruhn, A., Belyaev, A., Seidel, H.P.: Image compression with anisotropic di usion. Journal of Mathematical Imaging and Vision 31(2), 255{269 (Jul 2008). https://doi.org/10.1007/s10851-008-0087-0 [OpenAIRE]

13. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? In: CVPR (2012)

14. Griewank, A., Walther, A.: Algorithm 799: Revolve: An implementation of checkpointing for the reverse or adjoint mode of computational di erentiation. ACM Trans. Math. Softw. 26(1), 19{45 (Mar 2000) [OpenAIRE]

15. Guney, F., Geiger, A.: Deep discrete ow. In: Asian Conference on Computer Vision (ACCV) (2016)

41 references, page 1 of 3
Abstract
Modern optical flow methods are often composed of a cascade of many independent steps or formulated as a black box neural network that is hard to interpret and analyze. In this work we seek for a plain, interpretable, but learnable solution. We propose a novel inpainting based algorithm that approaches the problem in three steps: feature selection and matching, selection of supporting points and energy based inpainting. To facilitate the inference we propose an optimization layer that allows to backpropagate through 10K iterations of a first-order method without any numerical or memory problems. Compared to recent state-of-the-art networks, our modular CNN is ve...
Subjects
free text keywords: Optical flow, Feature selection, Black box (phreaking), Modular design, business.industry, business, Inpainting, Pattern recognition, Inference, Deep learning, Artificial neural network, Artificial intelligence, Computer science, Computer Science - Computer Vision and Pattern Recognition
Funded by
EC| HOMOVIS
Project
HOMOVIS
High-level Prior Models for Computer Vision
  • Funder: European Commission (EC)
  • Project Code: 640156
  • Funding stream: H2020 | ERC | ERC-STG
41 references, page 1 of 3

1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. PAMI (2012) [OpenAIRE]

2. Agresti, G., Minto, L., Marin, G., Zanuttigh, P.: Deep learning for con dence information in stereo and tof data fusion. In: The IEEE International Conference on Computer Vision (ICCV) Workshops (Oct 2017)

3. Barron, J.T., Poole, B.: The fast bilateral solver. ECCV (2016)

4. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Img. Sci. (2009) [OpenAIRE]

5. Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sciences (2010) [OpenAIRE]

6. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical ow estimation based on a theory for warping. In: ECCV (2004)

7. Butler, D.J., Wul , J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical ow evaluation. In: A. Fitzgibbon et al. (Eds.) (ed.) ECCV. pp. 611{625. Part IV, LNCS 7577, Springer-Verlag (Oct 2012)

8. Chen, T., Xu, B., Zhang, C., Guestrin, C.: Training deep nets with sublinear memory cost. CoRR abs/1604.06174 (2016)

9. Chen, Y., Ranftl, R., Pock, T.: A bi-level view of inpainting - based image compression. CoRR abs/1401.4112 (2014) [OpenAIRE]

10. Dollar, P., Zitnick, C.L.: Structured forests for fast edge detection. In: CVPR. pp. 1841{1848. ICCV '13, IEEE (2013) [OpenAIRE]

11. Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Haz rbas, C., Golkov, V., v.d. Smagt, P., Cremers, D., Brox, T.: Flownet: Learning optical ow with convolutional networks. In: ICCV (2015)

12. Galic, I., Weickert, J., Welk, M., Bruhn, A., Belyaev, A., Seidel, H.P.: Image compression with anisotropic di usion. Journal of Mathematical Imaging and Vision 31(2), 255{269 (Jul 2008). https://doi.org/10.1007/s10851-008-0087-0 [OpenAIRE]

13. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? In: CVPR (2012)

14. Griewank, A., Walther, A.: Algorithm 799: Revolve: An implementation of checkpointing for the reverse or adjoint mode of computational di erentiation. ACM Trans. Math. Softw. 26(1), 19{45 (Mar 2000) [OpenAIRE]

15. Guney, F., Geiger, A.: Deep discrete ow. In: Asian Conference on Computer Vision (ACCV) (2016)

41 references, page 1 of 3
Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue
publication . Other literature type . Part of book or chapter of book . Preprint . 2018

Learning Energy Based Inpainting for Optical Flow

Christoph Vogel; Patrick Knöbelreiter; Thomas Pock;