In 2014, the first dedicated Vision Processing Unit (VPU) was introduced, sparking an explosion in the market for edge AI products. Today, a wide variety of vendors offer products at different performance levels, ranging from ultra-low power speech AI chips to high-performance units that can seamlessly replace desktop GPUs. AI silicon architectures can be grouped into five broad categories: GPUs, streaming VPU architectures, dedicated solutions for verticals, programmable silicon products, and single vertical silicon. This report presents the benchmarks required for a novel state-of-the-art edge AI accelerator, as identified by potential customers and partners. A survey of the edge AI accelerator landscape is also presented, based on published benchmark data of competitors. In consultation with NeuroSoC partners, the report proposes specific neural network architectures for benchmarking the NeuroSoC chip based on the trends and needs of industry. The report also discusses the use cases influencing these choices and the rationale for choosing them. A set of industry based Key Performance Indicators (KPIs) are proposed to provide a benchmark for the NeuroSoC chip. On the software side, the requirements needed for NeuroSoC to be a competitive solution have also been outlined within this report. Overall, this report provides a comprehensive overview of the current landscape of edge AI accelerators and outlines the benchmarks and KPIs required for a novel state-of-the-art edge AI accelerator like the NeuroSoC chip. It also highlights the importance of a functional and easy-to-use toolchain for the chip to be competitive in the market.

This document contains information, which is proprietary to the NeuroSoC Consortium. Neither this document nor the information contained herein shall be used, duplicated or communicated by any means to any third party, in whole or in parts, except with prior written consent of the NeuroSoC consortium. It does not necessarily reflect the opinion of the European Union.