This paper details the development and training of a multimodal neural network designed for the analysis of optical and acoustic data streams. Solution described here is one of the user products developed as a result of NESTLER project, a Horizon 2020 project which proposes the implementation of an environmentally sustainable, integrated technological and information solutions for agriculture. The primary outcome of this research is the establishment of the Poultry Quality of Life Index. This index enables the use of advanced mathematical diagnostics to evaluate and forecast the health status of poultry flocks. Poultry Health Monitoring System, developed based on the Poultry Quality of Life Index, enables timely interventions to minimize damage from the negative impact of external factors. By leveraging the PQL Index, the system can accurately evaluate the health status of the flock and promptly identify signs of distress or potential threats. By continuously monitoring the flock's health and environmental conditions, the system can detect early signs of distress or potential threats. This proactive approach allows for immediate corrective actions, thereby reducing the impact of negative influences on the flock's wellbeingand overall productivity.