This paper proposes an approach to optimize the deployment of on-board video analytics for checking the correct positioning of luggage in aircraft cabins. The system consists of embedded cameras installed on top of the cabin and a heterogeneous embedded processor. Each camera covers multiple regions of interest (i.e., multiple seats or aisle sections) to minimize the number of cameras required. Each image region is processed by a separate image classification algorithm trained with the expected kind of visual appearance considering the effect of perspective and lens distortion. They classify these regions as correct or incorrect for cabin readiness by exploiting the hierarchical structure of classes composed of different configurations of passengers’ and objects’ presence or absence and the objects’ location. Our approach leverages semantic distances between classes to guide prototypical neural networks for multi-tasking between the main classification (i.e., correct or incorrect status) and auxiliary attributes (i.e., scene configurations), learning robust features from different data domains (i.e., various cabins, real or synthetic). The processing pipeline optimizes response delay and power consumption by leveraging embedded processors’ computing capabilities. We carried out experiments in a cabin mockup with a Jetson AGX Xavier, efficiently obtaining better-quality descriptive information from the scene to improve the system’s accuracy compared to alternative state-of-the-art methods.

This work was supported in part by the Clean Sky 2 Joint Undertaking through the European Union’s Horizon 2020 Research and Innovation Program under Grant 865162; in part by Smart Cabin System for cabin readiness COVID Amendment (SmaCS) (https://www.smacs.eu/), from the University of the Basque Country (UPV/EHU) under Grant GIU19/027; and in part by the Ministerio de Ciencia, Innovacion y Universidades, Agencia Estatal de Investigación (AEI), under Grant PID2021-126701OB-I00.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/

Peer reviewed