Humans are good at picking up statistical regularities in the environment. Probability cueing paradigms have demonstrated that the location of a target can be predicted based on spatial regularities. This is assumed to rely on flexible spatial priority maps that are influenced by visual context. We investigated whether stimulus features such as color distributions differing in mean and variance can cue location regularities. In experiment 1, participants searched for an oddly colored target diamond in a 6 × 6 set. On each trial, the distractors were drawn from one of two color distributions centered on different color averages. Each distribution was associated with different target location probabilities, one distribution where the target had an 80% chance to appear on the left (the rich location), while the rich location would be on the right for the other distribution. Participants were significantly faster at locating the target when it appeared in the rich location for both distributions, demonstrating learning of the relationship between color average and location probability. In experiments 2 and 3, observers performed a similar search task, but the distributions had different variances with the same average color. There was no evidence that search became faster when the target appeared in a rich location, suggesting that contingencies between target probabilities and color variance were not learned. These results demonstrate how statistical location learning is flexible, with different visual contexts leading to different spatial priority maps, but they also reveal important limits to such learning.