In Stackelberg security games, a defender must allocate scarce resources to defend against a potential attacker. The optimal defense involves the randomization of scarce security resources, yet how attackers perceive the risk given randomized defense is not well understood. We conducted an experiment where attackers chose whether to attack or not attack targets protected by randomized defense schemes, the key treatment variable being whether the defender picks one target at random to guard or imperfectly guards all targets. The two schemes are expected‐payoff equivalent, and when provided separately we found no effect of having one scheme or the other. Yet, when both are present, we found that subjects had a preference for the fixed scheme, a preference that cannot be reduced to differences in beliefs. Overall, our results suggest that understanding how individuals perceive risk is vital to understand the behavior of attackers.