To increase the quality of holographic correlation peaks, near stoichiometric Mg:Fe:LiNbO3crystals is used as memory and correlator due to its better uniformity and optical quality. A correlation peaks allocation algorithm based on neighborhood variance increment is presented, which is on the foundation of neighborhood entropy. This algorithm not only can distinguish target from edge texture, but also is not influenced by background intensity. With 1000 holograms stored, it takes less than 4 seconds to allocate correlation peaks accurately against correlation output image with 1024×768 pixels, and a satisfied recognition result is also achieved.