Abstract Vision-based robot tracking is commonly used for monitoring and debugging in single- and multi-robot environments. Currently, most of the existing vision-based multi-robot tracking systems are based on implementations on general purpose computers. These solutions are not feasible for embedded applications requiring high resource efficiency or high robustness as well as for use-cases with large frame sizes, multiple cameras and a large number of robots to be tracked. Field Programmable Gate Array (FPGAs)-based hardware accelerators can be used to efficiently handle compute-intensive applications like vision processing due to their high inherent parallelism. In this paper, we present an FPGA-based architecture for multi-robot tracking using multiple GigE Vision cameras. A complete system is implemented, comprising a multi-camera frame grabber and IP cores for image preprocessing, edge filtering, and circle detection. The robot localization is based on shape-based object detection. The proposed design is scalable in terms of the number of cameras and robots. It detects the locations of multiple robots simultaneously using single or multiple cameras without sacrificing the performance. Our implementation can process video frames from multiple cameras for multi-robot localization with precision and recall rates of 98%. It supports a maximum total video resolution of 2048x2048 with 152 frames per second. A speed-up of more than 30 is achieved as compared to a multi-threaded OpenCV implementation on a 3.2 GHz desktop quad-core CPU.