One function of perceptual systems is to construct and maintain a reliable representation of the environment. A useful strategy intrinsic to modern “Bayesian” theories of perception is to take advantage of the relative stability of the input and use perceptual history (priors) to predict current perception. This strategy is efficient but can lead to stimuli being biased toward perceptual history, clearly revealed in a phenomenon known as serial dependence. However, it is still unclear whether serial dependence biases sensory encoding or only perceptual decisions. We leveraged on the “surround tilt illusion”—where tilted flanking stimuli strongly bias perceived orientation—to measure its influence on the pattern of serial dependence, which is typically maximal for similar orientations of past and present stimuli. Maximal serial dependence for a neutral stimulus preceded by an illusory one occurred when the perceived, not the physical, orientations of the two stimuli matched, suggesting that the priors biasing current perception incorporate the effect of the illusion. However, maximal serial dependence of illusory stimuli induced by neutral stimuli occurred when their physical (not perceived) orientations were matched, suggesting that priors interact with incoming sensory signals before they are biased by flanking stimuli. The evidence suggests that priors are high-level constructs incorporating contextual information, which interact directly with early sensory signals, not with highly processed perceptual representations.