The magnitude of congruency effects, such as the flanker-compatibility effects, has been found to vary as a function of the congruency of the previous trial. Some studies have suggested that this congruency sequence effect is attributable to stimulus and/or response priming, and/or contingency learning, whereas other studies have suggested that the control process triggered by conflict modulates the congruency effect. The present study examined whether sequential modulation can occur without stimulus and response repetitions and contingency learning. Participants were asked to perform two color flanker-compatibility tasks alternately in a trial-by-trial manner, with four fingers of one hand in Experiment 1 and with the index and middle fingers of two hands in Experiment 2, to avoid stimulus and response repetitions and contingency learning. A significant congruency sequence effect was obtained between the congruencies of the two tasks in Experiment 1 but not in Experiment 2. These results provide evidence for the idea that the sequential modulation is, at least in part, an outcome of the top-down control process triggered by conflict, which is specific to response mode.