We propose in this article a new approach to robot cognitive control based on a stimulus-response framework that models both a robot’s stimuli and the robot’s decision to switch tasks in response to or inhibit the stimuli. In an autonomous system, we expect a robot to be able to deal with the whole system of stimuli and to use them to regulate its behavior in real-world applications. The proposed framework contributes to the state of the art of robot planning and high-level control in that it provides a novel perspective on the interaction between robot and environment. Our approach is inspired by Gibson’s constructive view of the concept of a stimulus and by the cognitive control paradigm of task switching. We model the robot’s response to a stimulus in three stages. We start by defining the stimuli as perceptual functions yielded by the active robot processes and learned via an informed logistic regression. Then we model the stimulus-response relationship by estimating a score matrix that leads to the selection of a single response task for each stimulus, basing the estimation on low-rank matrix factorization. The decision about switching takes into account both an interference cost and a reconfiguration cost. The interference cost weighs the effort of discontinuing the current robot mental state to switch to a new state, whereas the reconfiguration cost weighs the effort of activating the response task. A choice is finally made based on the payoff of switching. Because processes play such a crucial role both in the stimulus model and in the stimulus-response model, and because processes are activated by actions, we address also the process model, which is built on a theory of action. The framework is validated by several experiments that exploit a full implementation on an advanced robotic platform and is compared with two known approaches to replanning. Results demonstrate the practical value of the system in terms of robot autonomy, flexibility, and usability.