Our ability to focus on goal-relevant aspects of the environment is critically dependent on our ability to ignore or inhibit distracting information. One perspective is that distractor inhibition is under similar voluntary control as attentional facilitation of target processing. However, a rapidly growing body of research shows that distractor inhibition often relies on prior experience with the distracting information or other mechanisms that need not rely on active representation in working memory. Yet, how and when these different forms of inhibition are neurally implemented remains largely unclear. Here, we review findings from recent behavioral and neuroimaging studies to address this outstanding question. We specifically explore how experience with distracting information may change the processing of that information in the context of current predictive processing views of perception: by modulating a distractor’s representation already in anticipation of the distractor, or after integration of top-down and bottom-up sensory signals. We also outline directions for future research necessary to enhance our understanding of how the brain filters out distracting information.