Manipulating everyday objects without detailed prior models is still beyond the capabilities of existing robots. This is due to many challenges posed by diverse types of objects: Manipulation requires understanding and accurate model of physical properties of objects such as shape, mass, friction, elasticity, etc. Many objects are deformable, articulated, or even organic with undefined shape (e.g., plants) such that a fixed model is insufficient. On top of this, objects may be difficult to perceive, typically because of cluttered scenarios, or complex lighting and reflectance properties such as specularity or partial transparency. Creating such rich representations of objects is beyond current datasets and benchmarking practices used for grasping and manipulation. In this project we will develop an automated interactive perception pipeline for building such rich digitization. More specifically, in IPALM, we will develop methods for the automatic digitization of objects and their physical properties by exploratory manipulations. These methods will be used to build a large collection of object models required for realistic grasping and manipulation experiments in robotics. Household objects such as tools, kitchenware, clothes, and food items are not only widely accessible and in focus of many practical applications but also pose great challenges for robot object perception and manipulation in realistic scenarios. We propose to advance the state of the art by including household objects that can be deformable, articulated, interactive, specular or transparent, as well as shapeless such as cloth and food items. Our methods will learn physical properties essential for perception and grasping simultaneously from different modalities: vision, touch, audio as well as text documents such as online manuals and will include the following properties: 3D model, texture, elasticity, friction, weight, size and grasping techniques for intended use. At the core of our approach is a two-level modeling, where a category level model provides priors for capturing instance level attributes of specific objects. We will exploit online available resources to build prior category level models and a perception-action-learning loop will use the robot’s vision, audio, and touch to model instance level object properties. In return, knowledge acquired from a new instance will be used to improve the category-level knowledge. Our approach will allow us to efficiently create a large database of models for objects of diverse types, which will be suitable for example for training neural network based methods or enhancing existing simulators. We will propose a benchmark and evaluation metrics for object grasping, to enable comparisons of results generated with various robotics platforms on our database. The main objectives we pursue are commercially relevant robotics technologies, as endorsed by the support letters of several companies. We will pursue our goals with a consortium that brings together 5 world-class academic institutions from 5 EU countries (Imperial College London (UK), University of Bordeaux (France), Institut de Robòtica i informàtica Industrial (Spain), Aalto University (Finland), and the Czech Technical University (Czech Republic), assembling a complementary research team with strong expertise in the acquisition, processing and learning of multimodal information with applications in robotics.

MUSE-COM^2 aims to develop and validate a novel system for AI-empowered multimodal communications considering semantics of individual modalities jointly optimized with the processing of the modalities in multi-access/mobile edge computing (MEC) servers. Unlike conventional semantic and goal-oriented communications, the inclusion of information processing in MEC imposes new challenges related to the impact of information carried in individual modalities on the MEC processing outcome. The goal is to obtain a coherent framework jointly reducing the amount of information carried over the wireless links and subsequently processed in MEC; thus, saving not only radio and computing resources, but also energy while leading to the same outcome of the MEC processing. Since various modalities carry information about the same object, a redundancy in transmitted data for individual modalities will be identified and analyzed using AI. The AI training itself is typically a very complex and time/energy consuming process. Hence, we will also optimize the AI training to improve its efficiency and to reduce its energy consumption. Furthermore, we will design AI-based solutions managing jointly communication and computing considering the semantics of individual modalities to improve the overall system performance. The project targets a multi-disciplinary approach that combines elements of AI and communication systems. Consequently, our research approach will involve a combination of computational modeling, experimental studies, optimization, and data analysis. The developed system will be widely tested and validated in the lab and, then, the system will be demonstrated in the real-world environment in the facility of the industrial partner of the project. We target a practical industrial use-case focused on a production line monitoring and an evaluation of the products’ quality. The efficiency of the developed system will be assessed in terms of the targeted KPIs, especially energy savings for communications, computing, and AI training. MUSE-COM^2 will address challenges for traditional networks with respect to their design, deployment, operation and optimization to reduce power consumption related to communication and computing in line with the call. This will be facilitated via joint multimodal semantic communication and computing exploiting artificial intelligence and machine learning techniques implemented in wireless communication systems to improve network management and resource allocation. The following topics expected in the project call are targeted in MUSE-COM^2: - Design of AI-enhanced techniques for resource optimization in Radio Access Networks (addressed in Tasks T3.2, T3.3, and T4.2); - Implementation of ML in physical layer signal processing (T3.2, T3.3, and T4.2); - Development of software techniques to improve energy efficiency in wireless networks (T4.3); - Design of protocols for reliable AI-based Edge processing (T3.3 and T4.2); - Generation and assurance of reliable training data for ML (T3.1 and T4.1); - Development of open-access testbeds (T4.3 and T5.1); - Design of use-cases to take advantage of these technologies (T2.1). In line with the call, MUSE-COM2 will make all the data, protocol description, and software needed to reproduce experiments publically available.

Fungi constitute one of the largest groups of organisms on earth with central importance for ecosystem functioning. Despite their obvious relevance for understanding nature and ecosystem change, they have traditionally been neglected in conservation and monitoring, implying a wide-ranging knowledge gap. This project application has an overarching goal of closing this gap, by bringing fungi firmly on the biodiversity map. It will use existing citizen science data to explore spatiotemporal changes in fungal communities and analyse how well the Habitats Directive captures fungal biodiversity. Further, it will develop and test new tools and methods for fungal biodiversity mapping and monitoring, combining citizen science and standardized sampling of DNA from environmental sampling (eDNA). Finally, an important objective is to consolidate open data resources underlying collaboration on fungal biodiversity, by substantially improving taxonomic identification and data linked to DNA-based fungal occurrences. Overall, the project will hence address all three themes of the open Biodiversa call. The project is structured into clearly delegated, yet interlinked thematic work packages (WPs): 1. Improving identification of and unambiguous communication on fungal species 2. Applying and Improving AI tools for fungal monitoring 3. Involving citizen scientists in biodiversity discovery and monitoring 4. Sampling fungal communities by eDNA and 5. Analysing fungal biodiversity patterns in time and space. The project involves computer scientists, bioinformaticians, ecologists, taxonomists and citizen scientists collaborating to solve questions of societal interest. It is novel and seeking maximal applied impact by combining well-established, but so far isolated, tools in innovative ways. The consortium behind the project has a strong track record of previous collaborations and bridges research traditions in Northern, Central and Southern Europe, securing transfer of knowledge across regions, and a wide geographical scope on the ground for those WPs where this is central, i.e., WPs 1, 3 and 4. The project will not only provide a much-needed insight into the conservation status of fungi in Europe. Due to the critical roles fungi play in ecosystems, and their sensitivity to ecosystem change, improved insights into the fungal dimension of biodiversity will be of huge importance for understanding, more broadly, how global change affect ecosystems and associated ecosystem services mediated by fungi. Finally, we believe that the project will have impact on conservation and monitoring in other organism groups, by showcasing how molecular and AI methods in combination with unambiguous communication on species can be combined to increase credibility and impact of biodiversity data

The Second World War (WWII) is often considered the golden age of military aviation, but this air war has left a large number of remains in the European soil and in the sea: this massive commitment has caused considerable human and material losses. Even if WWII aircraft heritage has an undeniable historical and emotional value for Europeans, only recently these remains have officially entered the field of archaeology and cultural heritage conservation. Their presence in national museums is limited. They are often cared by numerous volunteers and associations. However, the discovery of an airplane wreck is challenging from several points of view: - its composition and materials, - its history, - its legal statutes, - its size and condition. PROCRAFT will face these challenges by connecting the multiple actors of the operational chain from recovery to exhibition. Scientists and associated partners (museums, associations, conservators, State representatives, mediators), from Italy, Czech Republic and France representing all the actors in this heritage chain, will pool and benefit from their joint expertise and capabilities. Our purpose is to create innovative procedures and solutions for each key step in aircraft conservation: - tailored techniques of conservation-restoration, - smart coatings for outdoor protection respecting the requirements of cultural heritage safeguard, - innovative solutions for preventive conservation in confined or semi-confined environments, - guidelines for Al alloys restoration and conservation for non-professional actors. The results of this project will: - enhance and share knowledge about conservation of WWII aircraft, focusing particularly on the conservation of aluminium (Al) alloy components, - contribute to its preservation, - promote its dissemination and presentation to the public.

MUSE-COM^2 aims to develop and validate a novel system for AI-empowered multimodal communications considering semantics of individual modalities jointly optimized with the processing of the modalities in multi-access/mobile edge computing (MEC) servers. Unlike conventional semantic and goal-oriented communications, the inclusion of information processing in MEC imposes new challenges related to the impact of information carried in individual modalities on the MEC processing outcome. The goal is to obtain a coherent framework jointly reducing the amount of information carried over the wireless links and subsequently processed in MEC; thus, saving not only radio and computing resources, but also energy while leading to the same outcome of the MEC processing. Since various modalities carry information about the same object, a redundancy in transmitted data for individual modalities will be identified and analyzed using AI. The AI training itself is typically a very complex and time/energy consuming process. Hence, we will also optimize the AI training to improve its efficiency and to reduce its energy consumption. Furthermore, we will design AI-based solutions managing jointly communication and computing considering the semantics of individual modalities to improve the overall system performance. The project targets a multi-disciplinary approach that combines elements of AI and communication systems. Consequently, our research approach will involve a combination of computational modeling, experimental studies, optimization, and data analysis. The developed system will be widely tested and validated in the lab and, then, the system will be demonstrated in the real-world environment in the facility of the industrial partner of the project. We target a practical industrial use-case focused on a production line monitoring and an evaluation of the products’ quality. The efficiency of the developed system will be assessed in terms of the targeted KPIs, especially energy savings for communications, computing, and AI training. MUSE-COM^2 will address challenges for traditional networks with respect to their design, deployment, operation and optimization to reduce power consumption related to communication and computing in line with the call. This will be facilitated via joint multimodal semantic communication and computing exploiting artificial intelligence and machine learning techniques implemented in wireless communication systems to improve network management and resource allocation. The following topics expected in the project call are targeted in MUSE-COM^2: - Design of AI-enhanced techniques for resource optimization in Radio Access Networks (addressed in Tasks T3.2, T3.3, and T4.2); - Implementation of ML in physical layer signal processing (T3.2, T3.3, and T4.2); - Development of software techniques to improve energy efficiency in wireless networks (T4.3); - Design of protocols for reliable AI-based Edge processing (T3.3 and T4.2); - Generation and assurance of reliable training data for ML (T3.1 and T4.1); - Development of open-access testbeds (T4.3 and T5.1); - Design of use-cases to take advantage of these technologies (T2.1). In line with the call, MUSE-COM2 will make all the data, protocol description, and software needed to reproduce experiments publically available.