The dot probe task is a widely used measure of attention allocation to threatening stimuli. The present two studies examine the reliability of different versions of this task using words as well as pictures as stimulus material. Estimates of both internal consistency and retest reliability over one week lead to the conclusion that the dot probe task is a completely unreliable measure of attentional allocation in non‐clinical samples. This unreliability may explain the inconsistent findings for the dot probe task as reported in the literature. Copyright © 2005 John Wiley & Sons, Ltd.