Assessing the escalating biodiversity crisis, driven by climate change, habitat destruction, and exploitation, necessitates efficient monitoring strategies to assess species presence and abundance across diverse habitats. Video-based surveys using remote cameras are a promising, non-invasive way to collect valuable data in various environments. Yet, the analysis of recorded videos remains challenging due to time and expertise constraints. Recent advances in deep learning models have enhanced image processing capabilities in both object detection and classification. However, the impacts on models’ performances and usage on assessment of biodiversity metrics on videos is yet to be assessed. This study evaluates the impacts of video processing rates, detection and identification model performance, and post-processing algorithms on the accuracy of biodiversity metrics, using simulated remote videos of fish communities and 14,406 simulated automated processing pipelines. We found that a processing rate of one image per second minimizes errors while ensuring detection of all species. However, even near-perfect detection (both recall and precision of 0.99) and identification (accuracy of 0.99) models resulted in overestimation of total abundance, species richness and species diversity due to false positives. We reveal that post-processing model outputs using a confidence threshold approach (i.e., to discard most erroneous predictions while also discarding a smaller proportion of correct predictions) is the most efficient method to accurately estimate biodiversity from videos.