Intelligent tracking of the behavior information of animal subject is important in multiple mode brain-machine-interface implementation in neuroscience research. This work proposes a low latency, low computation complexity image-based tracking system with a wireless neural interface. The proposed tracker integrated a light scale feature points recognition network and a prediction module for real-time location tracking. A detector is used to initialize the location of the target animal as well as to fix the error generated from the tracker. Experimental results shows a state-of-the-art processing rate of 117fps based on 2.4GHz, 20 cores CPU resources and a 166fps based on a GPU server resources.