• shareshare
  • link
  • cite
  • add
auto_awesome_motion View all 4 versions
Publication . Conference object . 2018

Attention-Enhanced Sensorimotor Object Recognition

Spyridon Thermos; Georgios Th. Papadopulos; Petros Daras; Gerasimos Potamianos;
Open Access
Published: 01 Oct 2018
Publisher: IEEE

Sensorimotor learning, namely the process of understanding the physical world by combining visual and motor information, has been recently investigated, achieving promising results for the task of 2D/3D object recognition. Following the recent trend in computer vision, powerful deep neural networks (NNs) have been used to model the “sensory” and “motor” information, namely the object appearance and affordance. However, the existing implementations cannot efficiently address the spatio-temporal nature of the humanobject interaction. Inspired by recent work on attention-based learning, this paper introduces an attention-enhanced NN-based model that learns to selectively focus on parts of the physical interaction where the object appearance is corrupted by occlusions and deformations. The model’s attention mechanism relies on the confidence of classifying an object based solely on its appearance. Three metrics are used to measure the latter, namely the prediction entropy, the average N-best likelihood difference, and the N-best likelihood dispersion. Evaluation of the attention-enhanced model on the SOR3D dataset reports 33% and 26% relative improvement over the appearance-only and the spatio-temporal fusion baseline models, respectively.

Subjects by Vocabulary

Microsoft Academic Graph classification: Affordance Cognitive neuroscience of visual object recognition Pattern recognition Feature extraction Task analysis Artificial intelligence business.industry business Entropy (information theory) Computer science Sensory system


Sensorimotor object recognition, attention mechanism, stream fusion, deep neural networks

Related Organizations
Funded by
EC| VRTogether
An end-to-end system for the production and delivery of photorealistic social immersive virtual reality experiences
  • Funder: European Commission (EC)
  • Project Code: 762111
  • Funding stream: H2020 | IA
Validated by funder
Download fromView all 3 sources
Conference object
License: cc-by
Providers: UnpayWall