Significance Individuality is a fundamental feature of behavior. For instance, the same smell or song may evoke very different responses in 2 individuals. What is the biological basis for these differences? While behavioral differences likely originate with differences in neural activity, little is known about how idiosyncratic behavioral differences are reflected in neural activity. We used statistical behavioral analysis and live brain imaging to assess idiosyncratic odor responses in fruit flies, and found that the same odors produce different behavioral responses across flies and that these odors evoke subtly different patterns of brain activity across flies. Moreover, neuromodulators and sets of neurons in the olfactory region of the fly’s brain directly modulate the degree of fly-to-fly behavioral variability in a flexible way.