Evolutionary dynamics depend critically on a population's interaction structure-the pattern of which individuals interact with which others, depending on the state of the population and the environment. Previous research has shown, for example, that cooperative behaviors disfavored in well-mixed populations can be favored when interactions occur only between spatial neighbors or group members. Combining the adaptive dynamics approach with recent advances in evolutionary game theory, we here introduce a general mathematical framework for analyzing the long-term evolution of continuous game strategies for a broad class of evolutionary models, encompassing many varieties of interaction structure. Our main result, the canonical equation of adaptive dynamics with interaction structure, characterizes expected evolutionary trajectories resulting from any such model, thereby generalizing a central tool of adaptive dynamics theory. Interestingly, the effects of different interaction structures and update rules on evolutionary trajectories are fully captured by just two real numbers associated with each model, which are independent of the considered game. The first, a structure coefficient, quantifies the effects on selection pressures and thus on the shapes of expected evolutionary trajectories. The second, an effective population size, quantifies the effects on selection responses and thus on the expected rates of adaptation. Applying our results to two social dilemmas, we show how the range of evolutionarily stable cooperative behaviors systematically varies with a model's structure coefficient.